Mining the Demographics of Political Sentiment from Twitter Using Learning from Label Proportions

Opinion mining and demographic attribute inference have many applications in social science. In this paper, we propose models to infer daily joint probabilities of multiple latent attributes from Twitter data, such as political sentiment and demographic attributes. Since it is costly and time-consuming to annotate data for traditional supervised classification, we instead propose scalable Learning from Label Proportions (LLP) models for demographic and opinion inference using U.S. Census, national and state political polls, and Cook partisan voting index as population level data. In LLP classification settings, the training data is divided into a set of unlabeled bags, where only the label distribution in of each bag is known, removing the requirement of instance-level annotations. Our proposed LLP model, Weighted Label Regularization (WLR), provides a scalable generalization of prior work on label regularization to support weights for samples inside bags, which is applicable in this setting where bags are arranged hierarchically (e.g., county-level bags are nested inside of state-level bags). We apply our model to Twitter data collected in the year leading up to the 2016 U.S. presidential election, producing estimates of the relationships among political sentiment and demographics over time and place. We find that our approach closely tracks traditional polling data stratified by demographic category, resulting in error reductions of 28-44% over baseline approaches. We also provide descriptive evaluations showing how the model may be used to estimate interactions among many variables and to identify linguistic temporal variation, capabilities which are typically not feasible using traditional polling methods

Co-training for Demographic Classification Using Deep Learning from Label Proportions

Deep learning algorithms have recently produced state-of-the-art accuracy in many classification tasks, but this success is typically dependent on access to many annotated training examples. For domains without such data, an attractive alternative is to train models with light, or distant supervision. In this paper, we introduce a deep neural network for the Learning from Label Proportion (LLP) setting, in which the training data consist of bags of unlabeled instances with associated label distributions for each bag. We introduce a new regularization layer, Batch Averager, that can be appended to the last layer of any deep neural network to convert it from supervised learning to LLP. This layer can be implemented readily with existing deep learning packages. To further support domains in which the data consist of two conditionally independent feature views (e.g. image and text), we propose a co-training algorithm that iteratively generates pseudo bags and refits the deep LLP model to improve classification accuracy. We demonstrate our models on demographic attribute classification (gender and race/ethnicity), which has many applications in social media analysis, public health, and marketing. We conduct experiments to predict demographics of Twitter users based on their tweets and profile image, without requiring any user-level annotations for training. We find that the deep LLP approach outperforms baselines for both text and image features separately. Additionally, we find that co-training algorithm improves image and text classification by 4% and 8% absolute F1, respectively. Finally, an ensemble of text and image classifiers further improves the absolute F1 measure by 4% on average

LIGHTLY SUPERVISED MACHINE LEARNING FOR CLASSIFYING ONLINE SOCIAL DATA

Classifying latent attributes of social media users has many applications in public health, politics, and marketing. For example, web-based studies of public health require monthly estimates of the health status and demographics of users based on their public communications. Most existing approaches are based on supervised learning. Supervised learning requires human annotated labeled data, which can be expensive and many attributes such as health are hard to annotate at the user level. In this thesis, we investigate classification algorithms that use population statistical constraints such as demographics, names, polls, and social network followers to predict individual user attributes. For example, the racial makeup of counties is a source of light supervision came from the U.S. Census to train classification models. These statistics are usually easy to obtain, and a large amount of unlabeled data from social media sites (e.g. Twitter) are available. Learning from Label Proportions (LLP) is a lightly supervised approach when the training data is multiple sets of unlabeled samples and only label distributions of them are known. Because social media users are not a representative sample of the population and constraints are too noisy, using existing LLP models (e.g. linear models, label regularization) is insufficient. We develop several new LLP algorithms to extend LLP to deal with this bias, including bag selection and robust classification models. Also, we propose a scalable model to infer political sentiment from the high temporal big data, and estimate the daily conditional probability of different attributes as a supplement method to polls, for social scientists. Because, constraints are not often available in some domains (e.g. blogs), we propose a self-training algorithm to gradually adapt a classifier trained on social media to a different but similar field. We also extend our framework to deep learning and provide empirical results for demographic classification using the user profile image. Finally, when both textual and profile image are available for a user, we provide a co-training algorithm to iteratively improve both image and text classifications accuracy, and apply an ensemble method to achieve the highest precision.Ph.D. in Computer Science, May 201